Managed Agents are easier to understand by starting with a local Agent. Once it needs to serve more users, run for longer, and recover reliably from component failures, its execution loop, durable record, and execution environment need to be separated so that each can be recovered and replaced independently.
This article is based on Anthropic's Scaling Managed Agents: Decoupling the brain from the hands. The travel-planning scenario is a running example used to explain the architecture.
Starting point: a local Agent that can complete a task
Building on Understanding Agent Harness Step by Step, imagine an AI travel assistant running locally:
Search for hotels in Shanghai, compare their prices against company travel policy, and produce a budget report.
The LLM decides what to do next. The Harness keeps the loop running, builds context, executes tool calls, and returns their results to the model. A browser searches, Python performs calculations, and the filesystem stores the report.
This is already the basic structure of an Agent. The next question is how to run many such tasks reliably as a SaaS service for many users.
The first multi-user change: isolated execution
A local assistant normally works in one person's environment. A SaaS Agent may simultaneously process User A's spreadsheet, User B's web pages, and User C's Python analysis.
If all tasks run in one execution environment without isolation, a task can exhaust shared resources, delete another task's files, or read data it should not access. For example, it might run rm -rf /workspace/*, access User B's files, or be manipulated by prompt injection into reading API keys.
The system therefore needs a boundary around code execution and file operations: the Agent's “hands” belong in a Sandbox.
A Sandbox is a constrained execution environment. A task can receive its own Sandbox, which isolates it from other tasks. It may be implemented with containers or other isolation technologies, but using a container alone does not complete the security design: file access, networking, permissions, and resources still need limits.
It is natural to ask whether the Harness should live there too. The first version can put the whole Agent in one container:
This is a reasonable SaaS Agent V1: one container per task, containing the Harness, tool execution, and temporary files. The container can be discarded when the task finishes. For short tasks, this is simple and effective. The problem appears as tasks run longer.
Long-running tasks make a container irreplaceable
Suppose an Agent has worked for an hour: it has searched dozens of hotels, downloaded company policy, compared prices, and is about to generate a report.
If conversation history, task progress, downloads, and Harness state exist only in that container, a crash can lose all of that work. The container becomes a special instance that must be carefully maintained.
This is the article's pets vs. cattle distinction:
- Pet: an instance holds irreplaceable state, so it has to be brought back when it fails.
- Cattle: an instance can be replaced because important state is stored and restored independently.
The important question is not how long a container runs, but whether the system can continue without that particular container.
Separating brains, hands, and Sessions
Anthropic's core design separates three responsibilities so they can run, be replaced, and scale independently:
| Component | Responsibility | Travel example |
|---|---|---|
| Brain: LLM + Harness | Decisions, the execution loop, context management, and tool routing | Decide to find hotels before calculating a budget |
| Hands: Sandbox + Tools | Perform concrete actions | Search the web, run Python, read and write files |
| Session | Durably record events that occur in a conversation | Store requests, tool calls, results, and errors |
The Harness uses an execution environment through a tool interface rather than relying on a particular machine. The article expresses this boundary as execute(name, input): the execution target might be a container, browser, or other remote environment. A failed brain or hand can be replaced; the Session durably preserves its event history.
Decoupling provides the foundation for recovery. The system must still persist events continuously and define how files and unfinished operations are restored.
A Session stores history; Context is the current working set
- Session: the system's durable event record.
- Context: the actual input visible to the model for one call.
- Context Engineering: selecting, organizing, and compressing information to build that input.
Suppose company policy limits hotels to 500 RMB per night, then the user adds “the hotel must have a gym.” If a summary retains only “find a suitable Shanghai hotel,” both constraints can be lost.
A durable Session lets the system retrieve the original events. The Harness decides when to retrieve them and how to place them into the finite context window. This connects to What I Learned About Context Engineering for AI Agents.
The relationship is similar to disk and memory: a Session is a durable record that can be revisited, while Context is the current working set. Persistence alone does not guarantee correct retrieval; a Harness can still omit important information.
Recovery requires knowing what was saved and how to continue
Once the Session is independent, a replacement Harness can read prior events and rebuild context after a crash. A Sandbox failure can return as a tool error, letting the system decide whether to provision another environment and continue.
However, recovering a Session record does not recover every execution state.
For example, an event that says “report generated” is not the report file itself. Downloaded material, file changes, browser state, and in-progress processes each need their own persistence, recovery, or recomputation strategy.
It is also important to distinguish rerunning from safe retrying. A price calculation can usually be redone. Retrying a hotel booking or sending an email when the first request may have succeeded can create a duplicate action.
Recovery therefore also needs mechanisms such as operation-status queries and idempotency keys. An idempotency key lets a supporting service recognize the same logical request and avoid executing it twice.
Decoupling enables on-demand resource allocation
An Agent can run for three hours without using Python, a browser, or file processing for all three hours. It may be waiting for model inference, an external service, or more information from the user.
After brains and hands are decoupled, a session that does not yet need code execution can begin reasoning before a Sandbox is created. The execution environment can be provisioned when needed and released when finished or idle. Anthropic reports that this architecture reduced its time to first token by roughly 60% at p50 and more than 90% at p95. In practical terms, p50 describes typical latency, and p95 describes a much slower tail case.
Pausing or releasing idle execution environments can also reduce SaaS costs, provided files and necessary state can be recovered. On-demand provisioning does not mean destroying an environment after every tool call; startup cost, caching, and task continuity matter too.
Security boundary: separate execution from credentials
Code running in a Sandbox may be influenced by untrusted web pages or files. If OAuth tokens and API keys are directly available there, a prompt injection can lead an Agent to read environment variables and expose credentials.
For MCP tools, the article keeps credentials in secure storage outside the Sandbox and uses a dedicated proxy to make external-service calls. The Harness never receives the raw credentials.
Hiding credentials addresses exposure, but the proxy must still check user identity, authorization scope, and the requested action. An Agent without a token can still make an inappropriate call through a tool it is allowed to use.
One Agent is not enough: orchestration and subagents
As travel planning becomes more complex, a Main Agent can split work between subagents that research flights and hotels. Each subagent has its own context, tools, and execution loop, so it can search, compare, and fill gaps before returning results to the Main Agent.
One Agent can also call tools in parallel. The value of a subagent is that it owns an independent task requiring multiple rounds of judgment, rather than merely issuing several queries at once.
For example, planning a Shanghai trip can first confirm dates from the calendar, then research flights and hotels in parallel, and finally have the Main Agent check timing and budget before producing options:
This is an extension of the Managed Agents architecture rather than the central pattern developed in the source article. Here, orchestration has two layers:
- Task Orchestration: performed by the Main Agent with its Harness. It decides how to break down the goal, which subagent receives each task, which tasks can run in parallel, and how to validate and combine results.
- Runtime Orchestration: performed by platform code. It starts Agents, allocates resources, controls concurrency, and handles pausing, timeouts, and recovery according to task requests and runtime policy.
For example, the Main Agent decides that separate flight and hotel Agents should research the request and provides dates, budget, and expected outputs. The platform runs those subagents. If the hotel Agent process crashes, the platform can recover it under an established policy without asking the Main Agent again. If it reports that no hotel meets the budget, the Main Agent normally decides whether to search again, adjust the plan, or ask the user.
Main Agents and subagents are therefore related by task delegation, while the platform manages the runtime of both. Each Agent's Harness maintains its own model and tool-calling loop. The Main Agent ultimately checks whether all results form a viable plan.
Multiple Agents increase model-call and coordination costs, so they are best used for tasks that can proceed independently and require their own deeper reasoning. A Harness lets one Agent keep acting; orchestration lets multiple Agents work together in an orderly way.
Mapping problems to components
| Problem | Responsibility or boundary introduced |
|---|---|
| The model must call tools repeatedly to complete a task | Harness / Agent loop |
| Multi-user code execution can interfere across users | Sandbox / permissions and resource isolation |
| A failed runtime instance loses task history | Persistent Session |
| History is too long for the model input | Context Engineering |
| Decision logic is coupled to the execution machine | Brain / Hands decoupling |
| A Sandbox is required before an execution need exists | On-demand execution provisioning |
| Untrusted code can read service credentials | Credential Vault / Tool Proxy |
| Multiple tasks and instances require coordination | Orchestration / subagents |
Final architecture: the core runtime of a SaaS Agent
The key shift is to consider both what an Agent can do and the conditions that let an Agent system run.
A Harness lets a model continue acting. A Managed Agent platform places that loop in a system that can isolate, recover, and scale it. As model capabilities change, the strategies encoded in a Harness also change. Managed Agents uses stable Session and execution interfaces so that those strategies can evolve without rebuilding the entire platform.
Source
Based on: Scaling Managed Agents: Decoupling the brain from the hands